TITLE OF THE INVENTION 

Image Pickup Apparatus Capable of Selecting Output According to 
Time Measured by Timer 
BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to image pickup apparatuses such as 
electronic still cameras and digital video cameras, and more particularly, to 
an image pickup apparatus for taking an image including only a 
background image but not an object and an image including the object. 
Description of the Background Art 

Japanese Patent Laying-Open No. 10-21408 discloses, for example as 
a conventional image pickup apparatus of this type, a technique of 
automatically extracting an object region from a moving image with stable 
accuracy and at a high speed independently of the shape of the object and 
the characteristic of image data by using a background image and an object 
image. 

In the conventional example, the object image and the background 
image are taken in the same mode so as to minimize a difference between 
image data corresponding to the background. After the images are taken, 
differential data of the background and object images is produced. Further, 
the differential data is averaged and subjected to a process utilizing a 



threshold value. Thereafter, the region of the object is extracted based on 
the difference between the image data of the background and object images. 
The extracted image data are compressed, encoded, and recorded. 

However, in general cases including the above mentioned example, a 
5 shutter must be pressed twice to input (take) background and object images. 
If an image pickup apparatus is fixed by a tripod or the like, the pickup 
apparatus slightly moves when pressing the shutter. This results in a 
problem that the background image and a portion of the object image 
corresponding to the background do not completely match. 

10 To avoid this problem, a background without any pattern, such as a 

plain colored wall, has often been used. In this case, an adverse effect due 
to the slight movement of the image pickup apparatus can be avoided. 
However, it is extremely difficult to prepare a plain colored background for 
normal, and especially outdoor picture taking. 

15 The image pickup apparatus may be designed such that the shutter 

can be pressed by a remote controller. However, this increases the cost of 
the image pickup apparatus and results in another problem that the remote 
controller must be carried on. 

Another possible approach is that the shutter is pressed by a release 

20 as in a normal optical camera. However, even if the shutter button would 
not be directly pressed, the image pickup apparatus may still be moved 
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slightly due to connection to the release, thereby causing another problem 
that a person other than the object is required for picture taking. 
SUMMARY OF THE INVENTION 

Therefore, an object of the present invention is to provide an image 
5 pickup apparatus preventing the movement of the apparatus when taking 
an image including only a background and that including an object. 

The image pickup apparatus according to the present invention takes 
a first image including only a background but not an object and a second 
image including an object, and has a shutter button which is pressed for 

10 releasing the shutter. The image pickup apparatus is characterized in 

that it includes" a time measuring portion measuring a period of time after 
the shutter button is pressed; and an output selecting portion outputting as 
one of the first and second images an image which has been taken when a 
first period of time is measured by the time measuring portion, and 

15 outputting as the other of the first and second images an image which has 
been taken when a second period of time is measured by the time 
measuring portion after the first period of time is elapsed. 

Thus, once the shutter button is pressed, the first and second images 
can be taken without touching the image pickup apparatus. 

20 Accordingly, a portion of a background of the second image and the 

first image match, so that the region of the object can be extracted with 
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high accuracy. In addition, since a remote controller is not required for 
pressing the shutter, the image pickup apparatus can be achieved with a 
low cost. Furthermore, since a plain colored background needs not be 
prepared, images can be taken under various conditions and flexibility of 
5 the image pickup apparatus is increased. 

In addition, since the picture-taking person can press the shutter 
button, he or she can be the object. Namely, picture taking can be 
performed only by the object. 

The image pickup apparatus of the present invention preferably 

10 includes • a region extracting portion outputting information indicating an 
object region of the second image by using the first and second images; and 
a recording portion recording onto a recording medium the data of 
positional information of the object region as well as the second image data 
or the image data included in the object region. 

15 Thus, the data indicating the object region is recorded during picture 

taking, so that the object region can be extracted after picture taking. 

Therefore, a user can always replace the background region of the 
second image with a desired image. More specifically, a desired image can 
be used as a background of the object image. 

20 The image pickup apparatus according to the present invention 

preferably includes^ a region extracting portion outputting positional 
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information on the object region in the second image by using the first and 
second images; an image composing portion replacing an image of a region 
other than the object region in the second image with a prepared 
background image! and a recording portion recording data of an image 
composed by the image composing portion onto a recording medium. 

Therefore, a user can always replace the background region of the 
second image with a desired image. 

Thus, the composed image can be verified as it is obtained. 

In addition, the image pickup apparatus according to the present 
invention preferably updates the first image by using the image of the 
region other than the object region in the second image every time a 
prescribed period is elapsed. 

As a result, the object region can be stably extracted even if the 
background image of the second image changes over time. 

In addition, the image pickup apparatus according to the present 
invention preferably includes a recording portion recording the first and 
second image data onto a recording medium. 

Thus, the region needs not be extracted in real time during picture 
taking, so that the burden on the image pickup apparatus can be reduced 
during picture taking. 

In addition, the image pickup apparatus according to the present 



invention preferably includes a notifying portion notifying a timing at 
which pickup of the first image is completed and a timing at which pickup 
of the second image is started. 

Thus, the image pickup states of the first and second images can be 
5 recognized by the user, so that the images are successfully taken. 

In the image pickup apparatus according to the present invention, 
preferably, the recording portion records the data in a compressed form. 

Thus, an amount of information recorded by the recording portion 
and that processed by the image pickup apparatus are reduced. 
10 BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing a schematic arrangement of an 
image pickup apparatus according to a first embodiment of the present 
invention. 

Figs. 2A and 2B are diagrams respectively shown for explaining 
15 background and normal images. 

Fig. 3 is a block diagram showing a region extracting portion of the 
image pickup apparatus shown in Fig. 1. 

Fig. 4 is a diagram shown in conjunction with an image pickup 
operation in the image pickup apparatus shown in Fig. 1. 
20 Fig. 5 is a block diagram showing a schematic arrangement of an 

image pickup apparatus according to a second embodiment of the present 
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invention. 

Figs. 6A to 6C are diagrams shown in conjunction with an operation 
of an image composing portion of the image pickup apparatus shown in Fig. 
5. 

Fig. 7 is a diagram showing a schematic arrangement of an image 
pickup apparatus according to a third embodiment of the present invention. 
DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Now, embodiments of an image pickup apparatus of the present 
invention will be described with reference to the drawings. 

First Embodiment 

Referring to Fig. 1, the image pickup apparatus of the present 
embodiment includes a CCD (Charge Coupled Device) 1, a timer 2 (a time 
measuring portion), and a mode selecting portion 3 (an output selecting 
portion). CCD 1 converts an optical image received by a lens (not shown) 
to an electric signal. Timer 2 measures a time after a shutter button is 
pressed in the image pickup apparatus. Mode selecting portion 3 switches 
between a mode of outputting a background image (a first image) not 
including an object to be extracted and a mode of outputting a normal 
image (a second image) including an object to be extracted in accordance 
with a time measured by timer 2. 

Referring to Fig. 2A, the background image refers to an image 



including only a background as denoted by image 2A. The background 
image is generally a still image, but may be an image obtained by taking an 
average of moving images within a prescribed period of time. 

Referring to Fig. 2B, the normal image refers to an image including 
an object 20 to be extracted (hereinafter referred to as "a target object") 
arranged in front of the background as denoted by image 2B. The normal 
image may be a still image or a moving image. 

It is noted that the image pickup apparatus of the present 
embodiment takes a still image as the background image and takes a 
moving image as the normal image. 

Returning back to Fig. 1, the image pickup apparatus of the present 
embodiment includes a background image buffer 4 and a region extracting 
portion 5. Background image buffer 4 stores a background image output 
from mode selecting portion 3. Region extracting portion 5 extracts a 
region with the target object (hereinafter referred to as "an object region") 
from the normal image output from mode selecting portion 3 and the 
background image stored in background image buffer 4 for outputting it as 
region information of binary or multilevel data. 

In addition, the image pickup apparatus of the present embodiment 
includes an image compressing portion 6, region information compressing 
portion 7, recording portion 8, and indicator 9. Image compressing portion 
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6 performs a general image compressing process on the normal image 
output from mode selecting portion 3. Region information compressing 
portion 7 compresses the region information output from region extracting 
portion 5 for outputting it as mask data. Recording portion 8 records the 
5 compressed normal image and region information onto a recording medium. 
Indicator 9 is attached in front of the image pickup apparatus so as to be 
visually recognizable to a person, e.g., an object. Indicator 9 is an 
exemplary notifying portion and designed to change its lighting state in 
accordance with a time measured by timer 2. It is for example a small red 

10 LED (Light Emitting Diode). 

It is noted that region extracting portion 5 includes a differentiator 
31, smoothing portion 32, and region determining portion 33, with reference 
to Fig. 3. Differentiator 31 calculates a difference between the normal and 
background images. Smoothing portion 32 smoothes the difference output 

15 from differentiator 31. Region determining portion 33 separates the 
regions in accordance with a prescribed threshold value for outputting 
region information of binary or multilevel data as the target object region. 

In other words, basically, region extracting portion 5 calculates the 
difference between the normal and background images. Then, when the 

20 difference between the background image shown as image 2A in Fig. 2A 
and the normal image shown as image 2B in Fig. 2B is calculated, ideally, 
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only the region with object 20 (the object region) has a difference with a 
large absolute value, whereas the difference in the peripheral region of 
object 20 is nearly 0. Thus, the object region can be separated. 

It is noted that a certain process is often performed after the 
5 difference is calculated in the actual procedure. Then, as described above, 
the separation of the region is performed based on the fact that the 
difference in the object region and that in the region other than the object 
region can be separated in accordance with a given threshold value. Thus, 
generally, as the difference in the region other than the object region comes 
10 close to 0, a separation performance increases. 

In addition, the image obtained by calculating the difference between 
the normal and background images tends to have noises near edges. 
Accordingly, the process of determining the region by using the threshold 
value is preferably performed after the smoothing process. Thus, the 
15 region can be extracted with higher accuracy. 

Next, the operation of the image pickup apparatus of the present 
embodiment will be described. 

The image received by CCD 1 is transmitted to mode selecting 
portion 3. Then, the image is output to background image buffer 4 in the 
20 background image pickup mode, but to region extracting portion 5 and 
image compressing portion 6 in the normal image pickup mode. 
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Here, referring to Fig. 4, the operation of the image pickup apparatus 
when the shutter is pressed will be described. Fig. 4 is a timing chart 
shown in conjunction with the operation after the shutter button is pressed 
in the image pickup apparatus, where an origin (a point of 0) of an abscissa 
5 corresponds to the point at which the shutter button is pressed. 

When a time Tl as measured by timer 2 is elapsed after the shutter 
button is pressed, the background image is taken. The picked up 
background image is output to background image buffer 4. Further, when 
a time T2 as measured by timer 2 is elapsed, pickup of the normal image is 
10 started. The picked up normal image is output to region extracting portion 
5 and image compressing portion 6. 

It is noted that the image pickup apparatus moves at the moment 
that the shutter button is pressed. Thus, time Tl must have a value 
greater than 0. Further, the object may be arranged in front of the 
15 background during time T2. In the image pickup apparatus, times Tl and 
T2 may arbitrarily be changed by a person taking the image depending on 
image pickup conditions. 

In image compressing portion 6, general image compression is 
performed, and compressed data of the normal image is recorded onto the 
20 recording medium by recording portion 8. In addition, region extracting 
portion 5 extracts the target object region from the normal and background 
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images, and region information indicating the object region is output to 
region information compressing portion 7. Then, the region information is 
recorded as mask data onto the recording medium by recording portion 8 
after being compressed by region information compressing portion 7. 
5 Accordingly, the object region can be extracted by using the mask 

data recorded in the recording medium and the compressed data of the 
normal image. Thus, the image of the background region (the region other 
than the object region in the normal image) can arbitrarily be replaced by 
an arbitrary background image for replacement at any point of time, for 

10 example when the normal image is displayed. 

As described above, in the image pickup apparatus of the present 
embodiment, the background image is taken at the point when time Tl is 
elapsed after the shutter button is pressed and the pickup of the normal 
image is started at the point when time (T1+T2) is elapsed. Namely, the 

15 person taking the image needs not touch the image pickup apparatus when 
the background and normal images are taken. In addition, the operation 
state of the image pickup apparatus automatically transfers from pickup of 
the background image to the normal image. 

Accordingly, the movement of the image pickup apparatus is 

20 prevented during pickup of the background and normal images, and the 

difference between the background image and the image in the background 
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region of the normal image can be reduced. Thus, the differential value of 
the background region can approximate to 0, so that the object region can 
be extracted with high accuracy. 

In addition, by changing the lighting pattern of indicator 9, the 
5 person to be the object can recognize the state of the image pickup 
apparatus. More specifically, the light of indicator 9 is turned on 
immediately after the shutter button is pressed. Then, the light of 
indicator 9 gradually starts turning on and off five seconds before the start 
of the pickup of the normal image, and more rapidly turns on and off three 
10 seconds before the start of the pickup. Thus, the person to be an object can 
prepare for the pickup more precisely. As a result, the pickup is reliably 
performed. 

Further, for the region which is determined the background region in 
region extracting portion 5, the background image stored in background 
15 image buffer 4 is preferably updated when a prescribed period of time is 
elapsed. Thus, an optimum background image can always be held in 
accordance with a change in background image over time. It is noted that 
the background image changes over time, for example when a slight change 
occurs to the image of the background region due to a change in 
20 illumination condition over time during outdoor picture taking. 

It is noted that the recording medium may record compressed data 
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including only an image in the object region of the normal image, instead of 
compressed data of the normal image. This is because, normally, a partial 
image is often processed as a set of compressed data and mask data of the 
image included in the object region. Namely, this facilitates utilization of 
5 the image of the object region as the partial image. 

It is noted that the image pickup apparatus of the present 
embodiment may be designed to recognize as the normal image the image 
taken at the point when time Tl is elapsed after the shutter is pressed, and 
recognize as the background image the image taken at the point when time 

10 T2 is further elapsed (at the point when time (T1+T2) is elapsed after the 
shutter is pressed), depending on its setting. In other words, such setting 
allows the image pickup apparatus to take the image including the object 
when the shutter is pressed and then to take the image including only the 
background and not the object. 

15 Second Embodiment 

Referring to Fig. 5, an image pickup apparatus of the present 
embodiment includes a CCD 1, timer 2, mode selecting portion 3, 
background image buffer 4, region extracting portion 5, image compressing 
portion 6, and recording portion 8 as components similar to those of the 

20 image pickup apparatus of the first embodiment. 

In addition, the image pickup apparatus of the present embodiment 
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includes a replacement image buffer 10 and image composing portion 11. 
Replacement image buffer 10 stores a background image for replacement. 
Image composing portion 11 replaces a region other than the object region 
of the normal image (background region) with the background image for 
5 replacement stored in replacement image buffer 10. 

The operation of the image pickup apparatus of the present 
embodiment will be described. The image received by CCD 1 is stored in 
background image buffer 4 as the background image in a background image 
pickup mode, but is output to region extracting portion 5 and image 
10 composing portion 11 as the normal image in the normal image pickup 
mode. 

Region extracting portion 5 extracts only the region of the target 
object from the background and normal images, and region information of 
binary or multilevel data, i.e., the extracted result, is output to image 
15 composing portion 11. Image composing portion 11 uses the region 
information from region extracting portion 5 to extract only the object 
region from the normal image, and replaces the other background region 
with the background image for replacement stored in replacement image 
buffer 10. 

20 For example, for image 6A which is an exemplary normal image 

shown in Fig. 6A, object 21 and background 22 are separated based on the 
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region information obtained from region extracting portion 5. Then, a 
portion of background 22 is replaced by background image for replacement 
background 6B shown in Fig. 6B, and a composed image 6C including object 
21 and background 23 as shown in Fig. 6C is output. 

The composed image which is output from image composing portion 
11 is compressed at image compressing portion 6, and then recorded in the 
recording medium by recording portion 8. More specifically, in the present 
embodiment, compressed data of the image with its background replaced is 
only recorded onto the recording medium, but mask data is not recorded. 
Thus, the amount of recording information can be reduced. In addition, 
the image with the replaced background image can be verified on the spot. 
It is noted that the background image for replacement is not limited to a 
still image and may be a moving image. 

Third Embodiment 

Referring to Fig. 7, an image pickup apparatus of the present 
embodiment includes a CCD 1, timer 2, mode selecting portion 3, and 
recording portion 8 as components similar to those of the image pickup 
apparatus of the first embodiment. 

Further, the image pickup apparatus of the present embodiment 
includes a background image compressing portion 12 and normal image 
compressing portion 13. Background image compressing portion 12 
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compresses the background image output from mode selecting portion 3 for 
outputting it to recording portion 8. Normal image compressing portion 13 
compresses the normal image output from mode selecting portion 3 for 
outputting it to recording portion 8. 

In the present embodiment, an object region is not extracted from the 
picked up background and normal images. Rather, the background and 
normal images are compressed and recorded onto the recording medium by 
recording portion 8. 

Namely, the objet region is not extracted in real time during the 
pickup operation. Thus, the amount of picture taking process is reduced. 
In this case, the compressed background and normal image data are 
decoded during reproduction, and the object region may be extracted by 
using them. Thus, a user can extract the object region by using an 
apparatus other than the image pickup apparatus such as a personal 
computer and the like. 

Although the present invention has been described and illustrated in 
detail, it is clearly understood that the same is by way of illustration and 
example only and is not to be taken by way of limitation, the spirit and 
scope of the present invention being limited only by the terms of the 
appended claims. 
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